Skip to content

Feat/multi tree forest - #114

Merged
nol4lej merged 2 commits into
mainfrom
feat/multi-tree-forest
Jul 31, 2026
Merged

Feat/multi tree forest#114
nol4lej merged 2 commits into
mainfrom
feat/multi-tree-forest

Conversation

@nol4lej

@nol4lej nol4lej commented Jul 30, 2026

Copy link
Copy Markdown
Member

Why

The pool ran a single depth-20 Merkle tree: 1,048,576 leaves, ever. The tree is append-only and spending a note doesn't free its leaf, so at ~500k private transfers MerkleTreeFull would have killed the network permanently. MERKLE_PROOF_SCALING_PLAN.md §5 explicitly deferred this decision; this PR is that decision, following MULTI_TREE_FOREST_PLAN.md (Railgun's model: seal a full tree, roll over to a new one).

What

Rollover. When an insert fills the tree it seals it in the same call: the final root goes to permanent storage, the frontier resets, and the next insert starts a fresh tree. The global u32 leaf index never resets (tree_id = leaf_index / MaxLeavesPerTree), which is what keeps event shapes, indexer chunking and wallet scan cursors untouched.

Sealed roots are permanent — this is the load-bearing part. The historic ring holds 100 roots and evicts on every insert. A naive port would have left a sealed tree's final root in there: 100 inserts into the new tree and every unspent note in the old one becomes permanently unspendable. That's not downtime, it's lost funds. So SealedTreeRoots / SealedRootIndex are separate permanent maps (bounded by tree count, max 4096), and is_known_root checks the ring OR the sealed set. The transfer/unshield/validate_unsigned call sites are unchanged — they already went through that one function.

No circuit, VK, extrinsic or ABI changes. Anchoring was already root-only (the chain never asks "which tree?"), and nullifier = Poseidon2(commitment, sk) carries no tree information, so cross-tree nullifier collisions aren't possible. The commitment-dedup and nullifier sets stay global across the forest — a duplicate commitment in a later tree would alias its nullifier and let one spend nullify both notes.

Config MaxLeavesPerTree (production 2^20, mocks 8 so rollover is testable). integrity_test enforces power-of-two ≤ 2^MAX_TREE_DEPTH: clients derive tree_id from this constant, so changing it on a live chain would re-map every note.

Runtime API v2 adds get_forest_info and get_root_for_leaf. The privacy RPC resolves each leaf's anchoring root through it, so a note in a sealed tree gets its permanent root instead of the live one — which also removes the root-drift retry problem for those notes. Response gains an additive tree_id.

STORAGE_VERSION 2 with a version-only migration (sealed maps start empty). Forest invariants merged into the existing try_state: active root always known, one sealed root per completed tree, sealed maps bijective.

Testing

Pallet suite 286 green (with --features try-runtime), precompile 65, RPC 35.

Rollover-specific unit tests: seal fires with the right event order (MerkleRootUpdated carrying the final root, then TreeSealed); the anti-freeze regression — a sealed root survives MaxHistoricRoots + N later inserts and its notes still verify; boundary straddling (leaves 7 and 8 land in different trees); duplicate commitment rejected across trees; multi-rollover with every leaf provable against its own tree's root; try_state holds across a seal.

Verified against a running node (dev runtime with an 8-leaf cap, scripts in ts-tests/): 20 shields seal trees 0 and 1 with distinct roots; all 20 leaves resolve to their own tree's anchoring root; lookup-by-commitment agrees with lookup-by-index on both root and path; sealed roots still correct after the later trees filled.

Security review

One real finding, fixed: the RPC proof handlers fell back to the active root when a leaf's anchoring root couldn't be resolved — that would have served a sealed-tree path with the wrong anchor. They now error explicitly. Fail-closed, not fail-wrong.

Verified as non-exploitable: proving against the empty root requires a Poseidon preimage; the size % cap == 0 boundary resolves to the sealed root; level-0 siblings can't cross a tree boundary (local ^ 1 stays inside); duplicate empty roots in the ring are covered by the existing eviction guard; the seal's ~4 uncharged writes are a once-per-million event, not a DoS vector.

Deploy notes

  • Consensus-affecting → spec_version 4 → 5. transaction_version unchanged.
  • The node binary requires runtime ≥ 5 (the RPC calls get_root_for_leaf). Upgrade the runtime before or together with the binaries.
  • Old SDK and app keep working: no extrinsic or ABI change, and with a single active tree the RPC returns exactly the root it returned before. Client-side forest support (same-tree coin selection) ships separately and only matters once a tree actually seals — roughly a million notes away.
  • Weights: the seal's extra writes were not folded into the benchmarked insert (~4 writes once per million inserts, noise against Groth16). Recovered on the next re-benchmark.

@nol4lej
nol4lej merged commit 0b24fa4 into main Jul 31, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant